Semi-supervised Feature Selection Research Articles

Adaptive orthogonal semi-supervised feature selection with reliable label matrix learning

Feature selection (FS) can select features with high value from high-dimensional data as much as possible and reduce the dimensionality of the data to improve the performance of the machine learning model and enhance the generalization ability. In semi-supervised FS (SSFS), regression-based methods utilize the label matrix, and the quality of the label matrix can directly affect the performance of FS. Hence, the importance of the reliable label matrix is obvious. Orthogonal regression (OR) can retain more information in the subspace than least squares regression (LSR). Therefore, this paper introduces OR into SSFS, and then the label scaling technique is used to learn a reliable label matrix. Also, this study utilizes adaptive graph learning to exploit more structural information about the data. Two constraints, Frobenius-norm or maximum information entropy imposed on the similarity matrix and two adaptive orthogonal SSFS (AGLSOFS) methods with reliable label matrix learning are constructed. The impact of these two constraints on the construction of dynamic similarity graphs and FS results is discussed. Effective optimization algorithms for these two methods are based on the Alternating Direction Method of Multipliers (ADMM) and Generalized Power Iteration (GPI). Experiments are conducted on 15 benchmark datasets, and the results show that: (1) similarity graphs constructed using both original and projected data are more accurate; (2) both constraints are valid; (3) both of the methods of this paper perform well on most datasets; (4) OR performs better than LSR for FS; and (5) the scaling factor affects the convergence speed of the model.

Self-paced semi-supervised feature selection with application to multi-modal Alzheimer’s disease classification

Semi-supervised multi-modal learning has attracted much attention due to the expense and scarcity of data labels, especially in disease diagnosis field. Most existing methods follow the paradigm by iteratively inferring the pseudo-labels of unlabeled data and add them into training sequence, but they ignore the reliability of those pseudo-labels, where inaccurate and wrong supervision will lead to negative influence on model learning. In this paper, we propose a Self-paced Semi-supervised Multi-modal Feature Selection (SSMFS) method, and apply it to Alzheimer’s disease classification. Specifically, SSMFS projects multi-modal biomedical data into the common label space with discriminative feature selection. Under the guidance of prior multi-modal similarity graphs, a unified graph is adaptively learned and embedded to preserve the neighborhood structures. More importantly, SSMFS dynamically investigates the discriminability and credibility of pseudo-labels, and adaptively assigns a weight to each unlabeled sample via self-paced learning such that the negative influence of wrong supervision can be reduced. Finally, a multi-kernel support vector machine is used to fuse the selected multi-modal features for final disease prediction. Experimental results on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset demonstrate the effectiveness of our method.

Semi-Supervised Feature Selection of Educational Data Mining for Student Performance Analysis

In recent years, the informatization of the educational system has caused a substantial increase in educational data. Educational data mining can assist in identifying the factors influencing students’ performance. However, two challenges have arisen in the field of educational data mining: (1) How to handle the abundance of unlabeled data? (2) How to identify the most crucial characteristics that impact student performance? In this paper, a semi-supervised feature selection framework is proposed to analyze the factors influencing student performance. The proposed method is semi-supervised, enabling the processing of a considerable amount of unlabeled data with only a few labeled instances. Additionally, by solving a feature selection matrix, the weights of each feature can be determined, to rank their importance. Furthermore, various commonly used classifiers are employed to assess the performance of the proposed feature selection method. Extensive experiments demonstrate the superiority of the proposed semi-supervised feature selection approach. The experiments indicate that behavioral characteristics are significant for student performance, and the proposed method outperforms the state-of-the-art feature selection methods by approximately 3.9% when extracting the most important feature.

SFS-AGGL: Semi-Supervised Feature Selection Integrating Adaptive Graph with Global and Local Information

As the feature dimension of data continues to expand, the task of selecting an optimal subset of features from a pool of limited labeled data and extensive unlabeled data becomes more and more challenging. In recent years, some semi-supervised feature selection methods (SSFS) have been proposed to select a subset of features, but they still have some drawbacks limiting their performance, for e.g., many SSFS methods underutilize the structural distribution information available within labeled and unlabeled data. To address this issue, we proposed a semi-supervised feature selection method based on an adaptive graph with global and local constraints (SFS-AGGL) in this paper. Specifically, we first designed an adaptive graph learning mechanism that can consider both the global and local information of samples to effectively learn and retain the geometric structural information of the original dataset. Secondly, we constructed a label propagation technique integrated with the adaptive graph learning in SFS-AGGL to fully utilize the structural distribution information of both labeled and unlabeled data. The proposed SFS-AGGL method is validated through classification and clustering tasks across various datasets. The experimental results demonstrate its superiority over existing benchmark methods, particularly in terms of clustering performance.

Semi-supervised feature selection with soft label learning

With the rapid increase of high-dimensional data mixed with labelled and unlabelled samples, the semi-supervised feature selection technique has received much attention in recent years. However, most existing approaches ignore the fuzziness of the data. Moreover, many feature selection methods need to measure the relationships among all samples, which is inefficient and difficult to be applied to large-scale data. To address the problems mentioned above, we propose an effective semi-supervised feature selection with the soft label learning (SFS-SLL) method in this paper. Specifically, we first learn initial soft labels based on the local distance between samples and clustering centers using an efficient fuzzy C-means clustering. We propose a supervised semantic constraint to exploit labelled and unlabelled data using manual labels as the soft label learning guidance. Then, we propose a simple yet effective sparse regression model which integrates soft label learning and feature selection into a unified framework. Finally, we derive an effective optimization strategy based on the alternating direction method of multipliers (ADMM) to iteratively solve the formulated problem. Experiment results on several benchmark datasets show a performance improvement on feature selection accuracy and efficiency over compared methods.

Semi-supervised feature selection based on fuzzy related family

Semi-supervised feature selection based on fuzzy related family

Efficient multi-view semi-supervised feature selection

Efficient multi-view semi-supervised feature selection

SemiFREE: Semisupervised Feature Selection With Fuzzy Relevance and Redundancy

Feature selection, as an effective dimensionality reduction technique, is favored in preprocessing data. However, most existing algorithms are solely liable for labeled or unlabeled data, whereas a limited portion of real-world data is annotated with labels. In this paper, we therefore propose a novel scheme named SemiFREE, i.e., Semi-supervised feature selection with Fuzzy RElevance and rEdundancy. Firstly, both labeled and unlabeled samples are assigned with fuzzy decisions that allow class membership to naturally express the fuzziness or uncertainty in data labeling. Secondly, sample similarities in feature space and fuzzy decision are captured to induce fuzzy information measures for redefining the feature relevance and redundancy. Finally, adhering to the principle of relevance-maximization and redundancy-minimization, SemiFREE leverages the forward sequential searching strategy to identify qualified features progressively. Extensive experiments demonstrate the superiority of SemiFREE in the presence of partially labeled data against some other well-established feature selection algorithms.

Semi-supervised multi-label feature selection algorithm for online monitoring of laser metal deposition manufacturing quality

Laser metal deposition (LMD) manufacturing, as a technique of laser metal additive manufacturing, has the advantages of short production periods, good economy and unrestricted forming shapes in the direct manufacturing of metal parts. However, the poor accuracy of single-object monitoring and the lack of labelled data make quality monitoring a huge challenge. Accordingly, this study extracts multi-features from the multi-object, including melt pool dimensions, spatters and temperature field, and proposes a semi-supervised multi-label feature selection (SSMLFS) algorithm. For SSMLFS algorithm, a multiple regression quality model is firstly proposed to produce different quality levels for unlabelled data based on microstructure. Then, to achieve feature dimension reduction and filter out highly relevant features, a quality correlation evaluation function is developed to calculate the contribution and ranking of the various features. Moreover, a local search algorithm based on quality is designed to improve the search speed of feature subsets and speed up the convergence of the SSMLFS algorithm. Experimental validation is conducted on BeAM Magic 800 machine and uses several commonly monitoring algorithms based on machine learning. Related experimental results prove that SSMLFS can significantly improve the accuracy and efficiency of quality monitoring compared to single-feature and full-feature monitoring. The proposed SSMLFS algorithm provides a semi-supervised feature selection framework to handle the low accuracy of single-feature monitoring and the complexity of multi-feature monitoring in the quality monitoring of LMD manufacturing.

Neighbourhood discernibility degree-based semisupervised feature selection for partially labelled mixed-type data with granular ball

Neighbourhood discernibility degree-based semisupervised feature selection for partially labelled mixed-type data with granular ball

Multi-filter semi-supervised transformer model for fault diagnosis

Dissolved Gas Analysis (DGA) is the most commonly used method for power transformer fault diagnosis. However, very few reliable and labeled fault DGA samples are available in the transformer substation whilst DGA data without labels is easier to obtain, which makes it difficult to train fault detectors in high-dimensional input space or select features using wrapper methods. Therefore, in order to improve the fault diagnosis accuracy using limited labeled DGA samples but more unlabeled DGA data, this paper proposes a novel multi-filter semi-supervised feature selection method for selecting optimal DGA features and building effective fault diagnosis models. A confidence criterion is also proposed for selecting high confidence unlabeled data to expand the training data set. Five filter techniques based on different evaluation criteria are employed to rank input DGA features, and a feature combination method is then applied to aggregate feature ranks by multiple filters and form a lower-dimensional candidate feature subset. The proposed method has been tested by using the IEC T10 dataset and compared with traditional supervised diagnostic models. The results show that the proposed method works well in optimizing DGA features and improving fault diagnosis accuracy significantly. Besides, the robustness of the selection of optimal feature subset is validated by testing DGA samples from the local power utility.

Iterative constraint score based on hypothesis margin for semi-supervised feature selection

To remove redundant features and avoid the curse of dimensionality, the most important features should be selected for downstream tasks, including semi-supervised learning. Several semi-supervised constraint scores using pairwise constraints have been proposed to estimate the relevance of features. However, these methods evaluate the features individually and ignore the correlations between them. Thus, we propose a semi-supervised feature selection method called the iterative constraint score based on the hypothesis margin (HM-ICS), which uses forward sequential selection to select an optimal feature subset with a good ability to maintain the constraint structure of the data and distinguish samples that belong to different classes. HM-ICS iteratively modifies the classical constraint score method to measure the relevance between features and maintain the constraint structure of the data. By introducing the hypothesis margin, HM-ICS can ensure strong discriminative power of the optimal feature subset. Extensive experiments were conducted on nine UCI and five high-dimensional datasets, and the experimental results confirmed that HM-ICS can achieve better performance than state-of-the-art supervised and semi-supervised methods.

Hessian-based semi-supervised feature selection using generalized uncorrelated constraint

Feature selection (FS) aims to eliminate redundant features and choose the informative ones. Since labeled data are not always easily available and abundant unlabeled data are often accessible, the importance of semi-supervised FS (SSFS), which selects the informative features utilizing the labeled and unlabeled data, becomes apparent. Semi-supervised sparse FS based on graph Laplacian (GL) regularization has become a popular technique. The GL regularization biases the solution towards a constant geodesic function, has a poor extrapolating ability, and cannot preserve well the topological structure of data. Traditional ridge regression is widely utilized by the GL-based semi-supervised sparse FS, but the results cannot gain the closed-form solution and carry out the manifold structure. To tackle these problems, we propose a Hessian-based SSFS framework using the generalized uncorrelated constraint (HSFSGU) and the mixed ℓ2,p-norm (0<p≤1) regularization. We also propose a Hessian–Laplacian-based SSFS framework using the generalized uncorrelated constraint (HLSFSGU) which utilizes the combination of GL and Hessian matrices. Our frameworks use the Hessian regularization for maintaining the topological structure of data well, the ℓ2,p-norm regularization for making the projection matrix appropriate for FS, and the generalized uncorrelated constraint for preventing exceeding row sparsity of the projection matrix and providing the elegant closed-form solution for our frameworks. We also present a unified algorithm to solve the frameworks for convex and non-convex cases, and prove its convergence. The results of the experiments depict the ability of the Hessian-based frameworks under generalized uncorrelated constraint in selecting the informative features.

A unified view of feature selection based on Hilbert-Schmidt independence criterion

Feature selection is a challenging and increasing important task in the machine learning and data mining community. According to different learning scenarios such as the availability of supervision information, feature selection can be broadly categorized as supervised, unsupervised and semi-supervised methods. Traditionally, feature selection algorithms are scenario-specific, i.e., different types of feature selection algorithms were proposed and investigated separately for specific learning scenario. In this paper, we aim to develop a unified view of supervised, unsupervised and semi-supervised feature selection methods. Based on the Hilbert-Schmidt independence criterion (HSIC), we first show that supervised, unsupervised and semi-supervised feature selection methods share the same objective. In discussing the connection between the unified framework and HSIC least absolute shrinkage and selection operator (HSIC Lasso), we find that the proposed framework not only has a clear statistical interpretation that minimum redundant features with maximum dependence on the target output variable can be found in terms of the HSIC, but also enables the global optimal solution to be computed efficiently by solving a Lasso optimization problem. Under this unified view, a new unsupervised feature selection algorithm is also proposed and demonstrated with several benchmark examples.

Automatic focal EEG identification based on deep reinforcement learning

Electroencephalogram (EEG) signals convey information about the electrical activity of neurons and are commonly used in clinical practice to evaluate the epileptic activity of patients. There is a part of the human brain that is closely linked to epileptic activity, namely the epileptogenic zone. Successful resection of the epileptogenic zone requires high precision classification of focal (F) and non-focal (NF) EEG signals. In this paper, we improve the Deep Q-Network (DQN) by adding Additional Functional Modules (AFM) to propose a novel focal EEG recognition method which is named AFM-DQN. Compared to the traditional Reinforcement Learning (RL), this model incorporates a deep convolutional neural network (CNN), and unifies the decision making capability of Q-learning and the perceptual capability of CNN, which greatly improves the learning of this network. The AFM, including pre-training, the high performance classifier (HPC), reward control mechanism and three RL related techniques, imparts this model a stronger generalization ability. To reduce computational burden of the network while preserving the correlation and interdependence of EEG signals, we introduce different kinds of EEG features and a semi-supervised feature selection algorithm based on joint mutual information (Semi-JMI algorithm). Experiments on the two publicly available EEG databases have achieved classification accuracy of 95.87% and 97.5%, respectively, which demonstrates that the proposed method has the potential for clinical application of accurate localization of epileptogenic zone. Compared with other published methods, this method has better performance and generalization ability.

Robust fuzzy rough approximations with kNN granules for semi-supervised feature selection

Fuzzy rough set theory has attracted much attention because of its successful application in uncertainty measurement. To improve the efficiency and robustness of uncertainty measure using the theory, robust fuzzy rough approximations have become popular research. In this work, it first designs a novel uncertainty measure based on kNN granules. By combining the relative distance, the method is effectively applicable to the datasets with large density variation. Using the measure a robust FRS model is studied and simply denoted KNN-FRS model. The new model takes a compromise between the maximum lower approximation degree and the minimum classification loss in computing fuzzy rough approximations, which reduces the sensitivity of the classical FRS model to noise. Besides, some properties of KNN-FRS model are summarized. Next, KNN-FRS model is used to design a feature selection algorithm which is tested to validate the feasibility of the KNN-FRS model. And then the KNN-FRS model is applied to design a semi-supervised feature selection algorithm which selects features while labeling samples and the labeling method is designed based on the proposed uncertainty measure. Some experiments are conducted to validate the KNN-FRS model and the designed algorithms. The results illustrate the proposed KNN-FRS model is feasible, effective and robust. And the designed semi-supervised feature selection algorithm performs excellent performance.

Semi-supervised feature selection for partially labeled mixed-type data based on multi-criteria measure approach

In many real applications, the data are always collected from different types and they are subjected to obtain partial labeling information of objects. Such data are referred to as partially labeled mixed-type data. There is currently few work on feature selection approaches for these data. Motivated by this issue, this paper aims at selecting the informative feature subset from partially labeled mixed-type data. At first, to improve the classification performance, an improved label propagation algorithm based on K-nearest neighbor is proposed, which marks the decision labels of unlabeled objects by making use of the information between unlabeled objects and labeled objects. On this basis, a feature multi-criteria measure based on the dependency, information entropy and information granulation is proposed for selecting candidate features. Finally, the corresponding semi-supervised feature selection algorithm is developed to select the feature subset for the partially labeled mixed-type data. Experimental results on UCI data sets demonstrate the effectiveness of the proposed feature selection algorithm and the superiority in terms of the classification accuracy compared with other algorithms.

Neurodynamics-driven holistic approaches to semi-supervised feature selection

Feature selection is a crucial part of machine learning and pattern recognition, which aims at selecting a subset of informative features from the original dataset. Because of label information, supervised feature selection performs better than unsupervised feature selection without label information. However, in the presence of a small number of labeled data and a large number of unlabeled data, it is challenging for supervised feature selection methods to select relevant features. In this paper, we propose three neurodynamics-driven holistic approaches to semi-supervised feature selection via semi-supervised feature redundancy minimization and semi-supervised feature relevancy maximization. We first define information-theoretic semi-supervised similarity coefficient matrix and semi-supervised feature relevancy vector based on multi-information, unsupervised symmetric uncertainty, and entropy to measure feature redundancy and relevancy. We then formulate a fractional programming problem and an iteratively weighted quadratic programming problem based on the semi-supervised similarity coefficient matrix and semi-supervised feature relevancy vector for semi-supervised feature selection. To solve the formulated problems, we delineate three neurodynamic optimization approaches based on two projection neural networks. We elaborate on the experimental results on six benchmark datasets to demonstrate the superior classification performance of the proposed neurodynamic approaches against six existing supervised and semi-supervised feature selection methods.

SemiACO: A semi-supervised feature selection based on ant colony optimization

Feature selection is one of the most efficient procedures for reducing the dimensionality of high-dimensional data by choosing a practical subset of features. Since labeled samples are not always available and labeling data may be time-consuming or costly, the importance of semi-supervised learning becomes apparent. Semi-supervised learning deals with data that includes both labeled and unlabelled instances. This article proposes a method based on Ant Colony Optimization (ACO) for the semi-supervised feature selection problem called SemiACO. The SemiACO algorithm finds features by considering the minimum redundancy between features and the maximum relevancy between the features and the class label. The SemiACO uses a nonlinear heuristic function instead of a linear one. The heuristic learning technique for the ACO heuristic function utilize a Temporal Difference (TD) reinforcement learning algorithm. We characterize the feature selection search space as a Markov Decision Process (MDP), where features indicate the states, and selecting the unvisited features by each ant represents a set of actions. We contrast the efficiency of SemiACO based on various experiments on 14 benchmark datasets, comparing eight semi-supervised feature selection methods.

Adaptive Data Structure Regularized Multiclass Discriminative Feature Selection.

Feature selection (FS), which aims to identify the most informative subset of input features, is an important approach to dimensionality reduction. In this article, a novel FS framework is proposed for both unsupervised and semisupervised scenarios. To make efficient use of data distribution to evaluate features, the framework combines data structure learning (as referred to as data distribution modeling) and FS in a unified formulation such that the data structure learning improves the results of FS and vice versa. Moreover, two types of data structures, namely the soft and hard data structures, are learned and used in the proposed FS framework. The soft data structure refers to the pairwise weights among data samples, and the hard data structure refers to the estimated labels obtained from clustering or semisupervised classification. Both of these data structures are naturally formulated as regularization terms in the proposed framework. In the optimization process, the soft and hard data structures are learned from data represented by the selected features, and then, the most informative features are reselected by referring to the data structures. In this way, the framework uses the interactions between data structure learning and FS to select the most discriminative and informative features. Following the proposed framework, a new semisupervised FS (SSFS) method is derived and studied in depth. Experiments on real-world data sets demonstrate the effectiveness of the proposed method.